Comparison of Overfitting and Overtraining
نویسندگان
چکیده
The application of feed forward back propagation artificial neural networks with one hidden layer (ANN) to perform the equivalent of multiple linear regression (MLR) has been examined using artificial structured data sets and real literature data. The predictive ability of the networks has been estimated using a training/ test set protocol. The results have shown advantages of ANN over MLR analysis. The ANNs do not require high order terms or indicator variables to establish complex structure-activity relationships. Overfitting does not have any influence on network prediction ability when overtraining is avoided by cross-validation. Application of ANN ensembles has allowed the avoidance of chance correlations and satisfactory predictions of new data have been obtained for a wide range of numbers of neurons in the hidden layer.
منابع مشابه
Boosting-like Deep Learning For Pedestrian Detection
This paper proposes boosting-like deep learning (BDL) framework for pedestrian detection. Due to overtraining on the limited training samples, overfitting is a major problem of deep learning. We incorporate a boosting-like technique into deep learning to weigh the training samples, and thus prevent overtraining in the iterative process. We theoretically give the details of derivation of our alg...
متن کاملHigh-dimensional dynamics of generalization error in neural networks
We perform an average case analysis of the generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant “high-dimensional” regime where the number of free parameters in the network is on the order of or even larger than the number of examples in the dataset. Using random matrix theory and exact solutions in linear models, we derive the gener...
متن کاملOvertraining and model selection with the self-organizing map
We discuss the importance of finding the correct model complexity, or regularization level, in the self-organizing map (SOM) algorithm. The complexity of the SOM is determined mainly by the width of the final neighborhood, which is usually chosen ad hoc or set to zero for optimal quantization error. However, if the SOM is used for visualizing the joint probability distribution of the data, then...
متن کاملOverfitting and Neural Networks: Conjugate Gradient and Backpropagation
Methods for controlling the bias/variance tradeoff typically assume that overfitting or overtraining is a global phenomenon. For multi-layer perceptron (MLP) neural networks, global parameters such as the training time (e.g. based on validation tests), network size, or the amount of weight decay are commonly used to control the bias/variance tradeoff. However, the degree of overfitting can vary...
متن کاملEvaluation of immune response after moderate and overtraining exercise in wistar rat
Objective(s): The effect of prolonged overtraining on cytokine kinetics was compared with moderate exercise in the present study. Materials and Methods: Male Wistar rats were randomly divided into control sedentary (C), moderate trained (MT), (V=20 m/min, 30 min/day for 6 days a week, 8 weeks), overtrained (OT) (V=25 m/min, 60min/day for 6 days a week, 11 weeks) and recovered overtrai...
متن کامل